METHOD OF AND APPARATUS FOR RETRIEVING MOVIE IMAGE 



RELATED APPLICATIONS 
[0001] This application relates to and claims a prior- 
ity from corresponding Japanese Patent Application No. 
2000-309364 filed on October 10, 2000. 



BACKGROUND OF THE INVENTION 
[0002] Field of the Invention 

The present invention relates to a method of and an 
apparatus for effectively retrieving or searching the 
movie image information for use in the multimedia informa- 
tion utilization field. 
[0003] Description of the Related Art 

Due to the fact that the computer is becoming high- 
speed and of large capacity in recent years, the data-base 
construction for the movie image information such as the 
movie and the video which have not been treated conven- 
tionally is becoming dramatically active. In accordance 
with this fact, techniques for effectively retrieving or 
searching a desired scene from a very large quantity of 
stored movie images have been put into practical. 
[0004] Such retrieval techniques for effectively se- 
lecting out the desired scene are classified largely into 
two methods- First method is one wherein indexes or key- 



words are assigned in advance to the movie image informa- 
tion and, at the retrieval operation, the user applies the 
key-word or search condition to the computer so that the 
desired movie image is detected. Second method is one 
wherein the brightness of color of the movie image is 
utilized as a key to detect the desired movie image. 
[0005] However, in the above method wherein the indexes 
or key-words are assigned in advance to the movie image 
information, there is a difficulty, for the user who has 
only ambiguous memory or insufficient information, in set- 
ting an appropriate search condition. Further, there is a 
problem in that search results themselves become incorrect 
depending on the memory or information that the user has 
or on the manner of the search key-words, 
[0006] In the second method wherein spatial signals 
such as the brightness or color of the images are used as 
keys, since the movie image information has greater quan- 
tity of data as compared to the text information or static 
image information, if the signal representing the movie 
image is subjected to the matching operation as it is, 
there occurs a problem in rendering the load on the hard- 
ware large and in increasing the time required for the 
searching process due to the large amount of information. 
[0007] With the above problems in the prior art taken 
into consideration, an object of the present invention is 
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to provide a method of and an apparatus for retrieving the 
movie image in which the necessary search is realized 
without depending on the memory or information that the 
user has and the manner of expression of the key-words, 
and in which the speed of the searching process is made 
high by decreasing the amount of information to be proc- 
essed. 

[0008] According to the invention, to solve the above 
problems, there is provided a method of retrieving the 
movie image, comprising the steps of: 

sequentially inputting, into a processor, the subject 
movie images from the movie image information comprising a 
number of successive images ; 

deriving feature values which vary in time from the 
signal of the inputted movie images; 

producing feature value information by quantization 
of the time feature value of the derived signal with a 
predetermined width of quantization; and 

matching, using a quantization error, the feature 
value information with the feature value information of 
the movie images stored in advance in the data-base. 
[0009] In this way, the subject movie image is time- 
sequentially inputted into the processor and, in the proc- 
essor, from the inputted movie image signals there is de- 
rived the feature values which vary in time. Then, the 
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derived time feature value of the signals is quantized 
with the predetermined width of quantization to produce 
the feature value information, and the feature value in- 
formation thus obtained is matched using the quantization 
error with the quantized time feature value of the movie 
image information stored in advance in the data-base. The 
feature value of the movie image information for a spe- 
cific scene is consecutive in time and there is a tendency 
that the value of the signal greatly varies when there 
occurs an abrupt change in the movie image or there occurs 
switching of the scenes. This can be detected by deriving 
the feature values which vary in time. Further, by quan- 
tizing the derived time feature value of the signals with 
the specific width of quantization, the region of wave is 
divided into finite number of small regions each region 
representing the specified value for the region. As a re- 
sult, the amount of data to be processed becomes small and 
thus the problem wherein the load on the hardware becomes 
large is effectively solved, and the shortening of the 
search processing time can be achieved. 
[0010] In another aspect of the invention, there is 
provided an apparatus for retrieving the movie image com- 
prising: 

an image input means for sequentially inputting, into 
a processor, the subject movie images from the movie image 
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information comprising a number of successive images; 

a feature value calculation means which comprises a 
feature value deriving section for deriving feature values 
which vary in time from the signal of the movie images 
inputted through the image input means, and a quantization 
process section for quantizing, with a predetermined width 
of quantization, the feature value derived from said fea- 
ture value deriving section; 

a comparative information selection means for deriv- 
ing, from a data-base that stores information in advance, 
comparative information corresponding to the movie image 
inputted through the image input means ; 

a matching process means for performing movie image 
matching using a quantization error between the feature 
value information obtained at the quantization process 
section in the feature value calculation means and the 
feature value information derived at the comparative in- 
formation selection means; and 

a search result process means for outputting the re- 
sult obtained at the matching process means. 

With this apparatus, the problem in which the load on 
the hardware becomes large has been solved and shortening 
of the processing time has been achieved. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0011] The above and other objects/ features and advan- 
tages of the present invention will be apparent from the 
following description of preferred embodiments of the in- 
vention explained with reference to the accompanying draw- 
ings, in which: 

Fig. 1 is a block diagram showing the basic principle 
of the present invention; 

Fig. 2 is a block diagram showing the hardware con- 
struction embodying the present invention; 

Fig. 3 is a block diagram showing the movie image 
searching processes executed in the CPU in Fig. 2; 

Fig. 4 is a diagram showing an example wherein the 
feature value of the movie image at the input side is de- 
rived and is quantized; 

Fig. 5 is a flow-chart showing the searching proce- 
dures according to the invention; 

Fig. 5 is a block diagram showing an embodiment of 
the movie image searching apparatus according to the in- 
vention; 

Fig. 7 is a block diagram showing an example wherein 
the feature value information is calculated from the lumi- 
nance information; 

Fig. 8 is a diagram showing an example of quantiza- 
tion of the luminance value as the time feature value; 
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Fig. 9 is a flow-chart showing the procedures of the 
embodiment of the invention; 

Fig. 10 is a block diagram showing an embodiment 
wherein the correlation calculated from the luminance 
value distribution is used as the feature value informa- 
tion; 

Fig. 11 is a diagram showing an example of the quan- 
tization of the correlation value calculated from the lu- 
minance value distribution as the time feature value; and 

Fig. 12 is a flow-chart showing the procedures of the 
embodiment of the invention* 

PREFERRED EMBODIMENTS OF THE INVENTION 
[0012] Now, embodiments according to the invention are 
explained with reference to the drawings. Fig. 1 is a 
block diagram showing the principle of the present inven- 
tion. 

[0013] Since the information (time series information) 
such as the movie image inf ormation or the audio informa- 
tion that has a time axis, that is, that changes in time 
sequence can be treated as waveform data, it is possible 
to determine whether an input data exists in the stored 
information by making the matching of the above waveform 
data with respect to the large amount of the stored 
information. 
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[0014] This invention enables a high-speed marching 
determination by obtaining the feature values from the 
time changing signal such as the movie image information 
and then the obtained feature value information is quan- 
tized with the predetermined width of quantization. With 
reference to Fig. 1, the movie image information at the 
movie image information input side A to which a search 
request is applied is inputted to one feature value calcu- 
lation means 1. In the calculation means 1, the feature 
value is obtained from the time changing image information 
and then is quantized with the specific width of quantiza- 
tion. In the same manner, the movie mage information at 
the data-base side B is inputted to the other feature 
value calculation means 2. In the calculation means 2, the 
feature value is obtained and is quantized with the spe- 
cific width of quantization. The feature value information 
thus obtained is inputted into the matching process means 
3 in which the matching is performed and from which the 
matching results are outputted. 

[0015] Here, assuming that the feature value at the 
move image information input side A is F x and that at the 
data-base side 3 is Fd/ the matching process between both 
the values is represented by the following Formula (1). 



(Fi - F d ) 2 £ Th 



(1) 



As represented by Formula (1), by determining both the 
feature value information with the threshold value Th 
which is the quantization error, it is possible to detect 
such time changing information as the movie image informa- 
tion and the audio information. In this invention, since 
the time changing feature values are effectively quan- 
tized, the amount of data subjected to the matching proc- 
ess is decreased, thereby enabling the high-speed search 
process . 

[0016] Now, the apparatus according to the present in- 
vention is explained in more detail. Fig. 2 is a block 
diagram showing a system structure embodying the inven- 
tion. Numeral 4 denotes a color display such as a CRT 
which displays an output of the computer 5. Commands or 
requests to the computer 5 are inputted through an input 
device 6 such as a keyboard or a mouse. Numeral 7 denotes 
a receiving line through which the search request informa- 
tion from the user's terminal device (not shown) is trans- 
mitted. 

[0017] in the computer 5 which has received the search 
request information through the input/output interface 8, 
the CPU 9 derives the feature value of the time changing 
image signal from the image information included in the 
search request information, and produces the feature value 
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information by being quantized with the specific width of 
quantization, in accordance with the programs stored in 
the memory 10. 

[0018] The computer 5 reads out the feature value in- 
formation in the data-base stored in the external memory 
device 12, performs the matching using the quantization 
error with the feature value information produced from the 
input image, and outputs the results thereof. The search 
result is displayed on the display device 4 or, if neces- 
sary, returned to the user's terminal (not shown) through 
the input/output interface 8 and the transmitting line 11 , 
that emitted the search request. Here, in the computer 5, 
in the case where the search of the image information 
within the user's terminal is to be effected without 
through the network, it is possible to conduct the search 
process of the movie image with the use of the in- 
put/output interface 8, 

[0019] Fig, 3 is a block: diagram showing the movie im- 
age searching processes performed in the CPU 9 in Fig. 2. 
The movie image searching method of the present invention 
is explained with reference to Fig. 3. 

[0020] In the computer 5, the image to be processed in 
the CPU 9 is read-in into the image input section 13 
through the input/output interface 8 in accordance with 
the program in the memory 10. Next, the signal of the 
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read-in movie image information is divided into two 
routes f one being the route A directed to the feature 
value calculation section 14 where the time feature value 
is obtained, and the other being the route B directed to 
the comparison information selection section 15 where the 
feature value information stored in the data-base to be 
matched with the above feature value information is se- 
lected. Specifically, the feature value deriving section 
16 in the feature value calculation section receives the 
image information from the image input section 13 and de- 
rives therefrom the signals of the brightness or the color 
that becomes the feature value of the input image. 
[0021] The derived information obtained at the feature 
value deriving section 16 is then inputted into the quan- 
tization process section 17 where the feature value is 
quantized with the specific width of quantization and is 
divided into a finite number of small regions, the infor- 
mation in each of those regions being represented by the 
specified value. The feature value information usable to 
the matching process is thus produced and then inputted to 
the matching process section 18. 

[0022] On the other hand, the comparison information 
selection section 15, in accordance with the image infor- 
mation inputted into the image input section 13, operates 
to select at the data-base side B the feature value infor- 
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mation which becomes the comparison information and which 
corresponds to the inputted image information. The feature 
value information thus selected is inputted to the match- 
ing process section 18. The matching process section 18 
receives the feature value information from the input side 
A and that from the data-base side B, and performs the 
matching operation on both the information. The result of 
this process is forwarded to the search result output sec- 
tion 19 which outputs the search result. 

[0023] Next, the quantization process section 17 which 
is a principal element in this invention is explained in 
detail with reference to Fig. 4. 

[0024] Fig. 4 is a graph which shows the changing lev- 
els in the direction of time, of the feature value of the 
image signal such as the brightness or the color of the 
movie image information. As shown in the drawings, the 
movie image information changes in its feature value in 
time for a given scene, and there is a tendency that the 
feature value largely changes in its level in the case 
where the image greatly changes or the scene is switched 
over from one to another. 3y utilizing the feature value 
of the image signal which varies in time, the width of the 
variation is quantized with the width T of quatization, 
whereby the representative value A of the feature value of 
the period L in direction of time is determined. Here, the 
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value A may be gained at the starting point or the ending 
point of the time period L, or it may be a mean value of 
the feature values in the same period L. Alternatively, 
the value A may be obtained by linear or non-linear divi- 
sion, for example, the peak or the center of the distribu- 
tion in the quantization period and, further, quantization 
accompanying equalizing or weighting may be adopted. 
[0025] The feature of the present invention is that, 
since time changing signal such as the brightness or color 
signal of the movie image information is utilized, any 
image sizes in the color space which can be processed by 
the computer can be utilized. 

[0026] Next, the searching procedures of the present 
invention are explained with reference to the flow-chart 
of Fig. 5. 

[0027] Step 101 through Step 105 are the processes for 
calculating the feature value information at the movie 
image information input side A. Step 106 through Step 110 
are the processes for calculating the feature value infor- 
mation at the data-base side B. The movie image informa- 
tion is inputted in Step 101, the feature value of the 
movie image information, which is used for the matching 
process, is calculated in Step 102, and the feature value 
information calculated is quantized with the width T of 
quantization in Step 103. Further, the period Li subjected 
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to the quantization is derived in Step 104, and the repre- 
sentative value Ai at the quatization period Li is derived 
in Step 105. On the other hand, the same procedures as 
above are performed at the data-base side B. In Step 110, 
the representative value A^ at the quantization period L d 
is derived- In this case, the feature value at the data- 
base side may have been calculated in advance with the 
process efficiency being taken into consideration. 
[0028] Further, in Step 111, the quantization period L x 
in the input side A and that Ld in the data-base side B are 
selected and, in Step 112, a determination is made as to 
whether Li and L d satisfy the Formula (1) . If YES, the 
process goes to Step 113 in which a determination whether 
Ai and Ad satisfy the Formula (1) is made. On the other 
hand, if NO, the process goes to Step 116 in which the end 
of matching is determined. 

[0029] In Step 113, the representative values Ai and A d 
of both the quantization periods are selected, and a de- 
termination as to whether the values A x and A d satisfy the 
Formula (1) in Step 114. If YES, the process goes to Step 
115 in which the result of matching is outputted. If NO, 
the process goes to Step 117 in which the end of matching 
is determined. 

[0030] Also, in Step 115, the result of matching is 
outputted and, in Step 116 a determination is made as to 
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whether the next L d exists or not. If YES, the process 
goes to Step 111 in which the next L d is selected and con- 
tinued to the matching process. If NO, the process goes to 
Step 115 in which the matching result is outputted. In 
Step 117, a determination as to whether the next L d exists 
is made. If YES, the process goes to Step 111 in which the 
next L d is selected and continued to the matching process. 
If NO, the process goes to Step 115 and the matching re- 
sult is outputted. 
[0031] First Embodiment 

Hereunder, some embodiments of the present invention 
are explained with reference to the accompanying drawings. 
Fig. 6 is a block diagram of the first embodiment which 
shows a movie image searching apparatus of the present 
invention. In this embodiment, the movie image information 
is inputted to the searching apparatus 24 from an input 
device, for example, a camera 20, a video player 21 and an 
external storage media 22. Here, the input device may be 
any type of device as far as it can process the movie im- 
age information. With the use of an input interface 23, 
the input of information from the network is also avail- 
able. The time feature value of the inputted movie image 
information is subjected to the quantization according to 
the method of the invention, the effective matching is 
then performed, the necessary information is derived from 
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the data-base 25 based on the search result, and the re- 
sult of the searching operation is provided to the user 
through the output interface 26 and from the output device 
such as the display device 27 and the external storage 
media 23. Here again, through the output interface 26, 
presentation of the search result using the network is 
available . 

[0032] As the time feature value of the movie image 
information, it is possible to use any information derived 
from the numerical picture element data such as color, 
luminance and its average value or distribution value of 
the movie image information, or distribution information. 
In this embodiment, as shown in Fig. 7 , the average value 
of the luminance signal is used as the time changing pa- 
rameter of the movie image information. Referring to Fig. 
7, the luminance value for each frame is obtained from the 
inputted movie image information and, then, the average 
value of the frame is calculated from the luminance value. 
By further quantizing the calculated average value, the 
quantization period and the representation value in that 
period are calculated. Fig. 8 is a graph showing the time 
changing aspect wherein the time feature value using the 
average value of the luminance value is subjected to 
quatization with the width T of quantization. Fig. 8 shows 
an example wherein the matching is performed using the 
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quantization periods Li through L 6 and their representative 
values Ai through A$ of the respective periods of the movie 
image information. 

[0033] If the luminance value of the image to be proc- 
essed is assumed to have an 8-bit precision, the luminance 
value for each picture element can be represented by a xy in 
the case where the size of the input image frame is x in 
vertical and y in horizontal. The average value of the 
luminance value in one (1) frame can be represented by the 
following Formula (2) . 

_ -J. 255 2hS 

a - — E Z a xy (2) 

x^O y=C 

[0034] The feature value information is produced by 
obtaining the average values for the respective frames and 
then these average values are quantized with the quantiza- 
tion width T. This feature value information includes the 
quatization periods Li through L$ and the representative 
values Ai through A 6 shown in Fig. 8. In the same manner 
as above, the feature value information at the data-base 
side is produced and is compared with the respective val- 
ues. Specifically, a determination between, for example, 
the quantization period Li at the input side and the quan- 
tization period L<j at the data-base side and, in the same 
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manner, a determination between, for example, the repre- 
sentative value As. in that quantization period Li at the 
input side and the representative value A d in that quanti- 
zation period L d at the data-base side are performed using 
the following Formulas (3) and (4). 

(Li - L d ) 2 ^ Th (3) 

(Aj. - A d ) 2 £ Th (4) 

[0035] Next, the procedures of this embodiment are ex- 
plained with reference to the flow-chart of Fig. 9. 

[0036] Step 201 through Step 206 are the processes for 
calculating the feature value information at the movie 
image information input side A. Step 207 through Step 210 
are the processes for calculating the feature value infor- 
mation at the data-base side B. The movie image informa- 
tion is inputted in Step 201, the luminance value of the 
movie image information is derived in Step 202, the aver- 
age value is calculated from the derived luminance value 
in Step 203, and the quantization of the average value of 
the luminance value obtained in Step 203 is made in Step 
204. In Step 205, the values of the quantization periods 
Li through L 6 are obtained and, in Step 206, the values of 
the representative values A x through A 6 corresponding to 
the quantiztion periods Li through U are obtained. On the 
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other hand, the similar procedures are performed at the 
data-base side B . Specifically/ the movie image informa- 
tion at the data-base side is inputted in Step 207, the 
luminance value of the movie image information is derived 
in Step 208, the average value is calculated from the de- 
rived luminance value in Step 209, and the quantization of 
the average value of the luminance value obtained in Step 
209 is made in Step 210. Up to the above steps at the 
data-base side, the feature values may have been calcu- 
lated in advance with the process efficiency being taken 
into consideration. 

[0037] Further, in Step 211, the quantization period L d 
at the data-base side B is selected and, in Step 212 a 
determination is made as to whether Li and L d satisfy the 
Formula (3) ♦ If YES, the process goes to Step 213 in which 
the period L d +i is selected. On the other hand, if NO, the 
process goes to Step 236 in which the end of matching is 
determined, 

[0038] In Step 213, the quantization period L d+ i at the 
data-base side B is selected and, in Step 214 a determina- 
tion is made as to whether L 2 and L d+ i satisfy the Formula 
(3) . If YES, the process goes to Step 215 in which the 
period L d+2 is selected. On the other hand, if NO, the 
process goes to Step 237 in which the end of matching is 
determined. 
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[0039] In Step 215, the quantization period L d+2 at the 
data-base side B is selected and, in Step 216 a determina- 
tion is made as to whether L 3 and L d+ 2 satisfy the Formula 
(3) . If YES, the process goes to Step 217 in which the 
period Ld+3 is selected. On the other hand, if NO, the 
process goes to Step 233 in which the end of matching is 
determined. 

[0040] In Step 217, the quantization period L d+3 at the 
data-base side B is selected and, in Step 218 a determina- 
tion is made as to whether L 4 and L d +3 satisfy the Formula 
(3) , If YES, the process goes to Step 219 in which the 
period is selected. On the other hand, if NO, the 
process goes to Step 239 in which the end of matching is 
determined, 

[0041] In Step 219, the quantization period L d+4 at the 
data-base side B is selected and, in Step 220 a determina- 
tion is made as to whether Ls and L d+4 satisfy the Formula 
(3) . If YES, the process goes to Step 221 in which the 
period L d +s is selected. On the other hand, if NO, the 
process goes to Step 240 in which the end of matching is 
determined. 

[0042] In Step 221, the quantization period L d+ s at the 

data-base side B is selected and, in Step 222 a determina- 
tion is made as to whether L$ and L d+5 satisfy the Formula 
(3) . If YES, the process goes zo Step 223 in which the 
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representative value A d in the quantization period L d is 
selected. On the other hand, if NO, the process goes to 
Step 241 in which the end of matching is determined. 
[0043] In Step 221/ the representative value A d in the 
quantization period L d at the data-base side B is selected 
and, in Step 224 a determination is made as to whether Ai 
and Ad satisfy the Formula (4). If YES, the process goes 
to Step 225 in which the value A d *i is selected. On the 
other hand r if NO, the process goes to Step 242 in which 
the end of matching is determined. 

[0044] In Step 225, the representative value Ad+i at the 
data-base side B is selected and, in Step 226 a determina- 
tion is made as to whether A 2 and Ad+i satisfy the Formula 
(4) . If YES, the process goes to Step 227 in which the 
value Ad+2 is selected. On the other hand, if NO, the proc- 
ess goes to Step 243 in which the end of matching is de- 
termined. 

[0045] In Step 227, the representative value A d *2 at the 
data-base side B is selected and, in Step 228 a determina- 
tion is made as to whether A3 and A d +2 satisfy the Formula 
(4). If YES, the process goes to Step 229 in which the 
value A<i + 3 is selected. On the other hand, if NO, the proc- 
ess goes to Step 244 in which the end of matching is de- 
termined. 

[004 6] In Step 229, the representative value A d +s at the 



data-base side B is selected and, in Step 230 a determina- 
tion is made as to whether A4 and A d + 3 satisfy the Formula 
(4) . If YES, the process goes to Step 231 in which the 
value Ad+4 is selected. On the other hand, if NO, the proc- 
ess goes to Step 245 in which the end of matching is de- 
termined. 

[0047] In Step 231, the representative value A d+4 at the 
data-base side B is selected and, in Step 230 a determina- 
tion is made as to whether A 5 and A d+4 satisfy the Formula 
(4). If YES, the process goes to Step 233 in which the 
value Ad+s is selected. On the other hand, if NO, the proc- 
ess goes to Step 24 6 in which the end of matching is de- 
termined. 

[0048] In Step 233, the representative value A d+5 at the 
data-base side B is selected and, in Step 234 a determina- 
tion is made as to whether A 6 and A d+ s satisfy the Formula 
(4) . If YES, the process goes to Step 235 in which the 
result of matching is outputted. On the other hand, if NO, 
the process goes to Step 247 in which the end of matching 
is determined. 

[0049] In Step 235, the matching result as to whether 
the determination formula is satisfied is outputted. 
[0050] In the Step 236 through Step 241, a determina- 
tion is made as to whether the next quantization period L d , 
Ldw./ L d ^2/ Ld+3r L d+4 or L d+ $ does exist or not. If YES, the 
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process goes to Step 211 in which the matching is contin- 
ued in the next quantization period L d and, if No, the 
process goes to Step 235 in which the result of the match- 
ing is outputted. 

[0051] In the Step 242 through Step 247, a determina- 
tion is made as to whether the next representative value 
Ad, Ad+i f Aa+2/ A d+3 , Ad+a or A d+ 5 does exist. If YES, the 
process goes to Step 211 in which the matching is contin- 
ued in the next quantization period L d and, if No, the 
process goes to Step 235 in which the result of the match- 
ing is outputted. 

[0052] As explained above, it is possible to achieve 
the high-speed searching by conducting the matching using 
the length of the quantization period as a pattern. In the 
same way, it is also possible to achieve the desired 
searching by conducting the matching using the representa- 
tive value. Here, as explained before, in the matching of 
the feature value information between the input side and 
the data-base side, all the steps are not necessarily per- 
formed- Matching result up to the intermediate step may 
well be used, if necessary. Further, the matching may well 
be such one as the combination of the quantization periods 
and the representative values, or such one as the partial 
combination of the quantization periods of the longest 
one. According to the present invention, by giving chang- 
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ing width or allowance to the quantization width T and the 
quantization period length L as well as the representative 
values at the respective quantization periods, it is pos- 
sible to perform the search for the movie image informa- 
tion even in the case where the image size of the input 
image which is inputted as the search process condition is 
different from the image size of the image at the data- 
base side, or where the coding rate which is one factor to 
determine the amount of information is incorrect in the 
image compression technique which is used for reducing the 
amount of information of the image data. 
[0053] Second Embodiment 

Next, another embodiment of the invention is ex- 
plained. In this embodiment, as the time feature value of 
the movie image information, amplitude distribution of the 
luminance signal between before and after the frame of the 
movie image information, that is, the frequency distribu- 
tion of the luminance signal is used and its correlation 
is utilized. Fig. 10 is a block diagram showing an embodi- 
ment wherein the correlation value calculated from the 
luminance value distribution is used as the feature value 
information. Referring to Fig. 10, first, the luminance 
signal is calculated from the inputted movie image infor- 
mation, the distribution of the amplitude thereof is ob- 
tained, then, the correlation value between before and 
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after the frame from the above amplitude distribution in- 
formation is calculated, and the quantization period and 
the representative values in the quantization period are 
calculated by the quantization of the correlation value* 
Fig. 11 is a graph showing the case wherein the correla- 
tion value calculated from the luminance value distribu- 
tion is quantized as the time feature value. Since the 
feature value information of the inputted movie image in- 
formation is the quantization of the correlation value, 
the time changing where the quantization is made with the 
quantization width T becomes the waveform as shown in Fig, 
11. In Fig* 11, the matching is conducted with the longest 
quantization period L7 among the quantization periods of 
the inputted movie image information, the quantization 
period L« which is before the quatization period L7 by one, 
the quantization period L Q which is after the quatization 
period L7 by one, and the representative values A6 and Ae 
in both the quantization periods L € and L 8 * 
[0054] If the luminance value of the image to be proc- 
essed here is assumed to have an 8-bit precision/ first, 
the frequency distribution of the luminance values for the 
frames of the inputted image is obtained and, then, the 
correlation values between before and after the frame us- 
ing the frequency distribution are obtained for the re- 
spective frames . 
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[0055] 



Specifically, assuming that the frequency dis- 



tribution for the i-order frame is a and the frequency 
distribution for the (i+1) -order fame is 0 , the correla- 
tion C can be obtained from the following Formula (5) . 



In this way, the feature value information is produced by 
being quantized / with the quantization width T, the corre- 
lation value C calculated from between the before frame 
and the after frame. The feature value information in- 
cludes the quantization periods Li through Lu and the rep- 
resentative values Ai through An. Similarly, the feature 
value information at the data-base side is produced. These 
values are subjected to the comparison operation. Namely, 
the quantization periods Li and Ld are determined using the 
following Formula (6), and the quatization period repre- 
sentative values At and Ad are determined using the follow- 
ing Formula (7) . 




(5) 



Here, a 




(Li - L d ) 2 ^ Th 
(Ai - A d ) 2 ^ Th 



. (6) 
. (7) 
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[0056] Next, the procedures of this embodiment are ex- 
plained with reference to the flow-chart of Fig. 12. 
[0057] Step 301 through Step 310 are the processes for 
calculating the feature value information at the movie 
image information input side A. Step 311 through Step 315 
are the processes for calculating the feature value infor- 
mation at the data-base side B. The movie image informa- 
tion is inputted in Step 301, the luminance value of the 
movie image information is derived in Step 302, the dis- 
tribution information is calculated in Step 303 from the 
derived luminance value in Step 302 , and in Step 304 the 
correlation value of the luminance distribution before and 
after the frame is calculated. In Step 305, the quantiza- 
tion of the correlation values obtained in Step 304 is 
made. 

[0058] In Step 306, the quantization period L 7 which is 
the longest one among the quantization periods is se- 
lected; in Step 307, the quantization period L 6 which is 
positioned before the quantization period is selected; 
and in Step 308, the quantization period Le which is posi- 
tioned after the quantization period L? is selected- Next, 
in Step 309 the representative value A s in the quantization 
period L € is selected and, in Step 310, the representative 
value A3 in the quantization period Le is selected. 

[0059] On the other hand, in Step 311, the movie image 
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information at the data-base side is inputted and, in Step 
312, the luminance value of the inputted movie image in- 
formation is calculated. Then, in Step 313, the distribu- 
tion information is calculated from the luminance value 
obtained in Step 312. Further, in Step 314, the correla- 
tion value of the luminance distribution between before 
and after the frame is calculated. In Step 315, the quan- 
tization of the correlation values obtained in Step 314 is 
made. Up to the above steps at the data-base side, the 
fearure value information may have been calculated in ad- 
vance with the process efficiency being taken into consid- 
eration. 

[0C60] In Step 316, the quantization period L d at the 
data-base side is selected and, in Step 317, a determina- 
tion is made as to whether L 7 and L d satisfy the Formula 
(6). If YES, the process goes to Step 318 in which the 
period L d -i is selected. If NO, the process goes to Step 

327 in which the end of matching is determined. 

[0061] In Step 318, the quantization period L d -i at the 
data-base side is selected and, in Step 319, a determina- 
tion is made as to whether Lq and Ld-i satisfy the Formula 

(6) . If YES, the process goes to Step 320 in which the 
period L d+ i is selected. If NO, the process goes to Step 

328 in which the end of matching is determined. 

[0062] In Step 320, the quantization period L d +i at the 
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data-base side is selected and, in Step 321, a determina- 
tion is made as to whether L a and L d +i satisfy the Formula 
(6), If YES, the process goes to Step 322 in which the 
representative value Ad-i in the period L d -i is selected. If 
NO, the process goes to Step 329 in which the end of 
matching is determined. 

[0063] In Step 322, the representative value A d -i in the 
quantization period Ld-i at the data-base side is selected 
and, in Step 323, a determination is made as to whether As 
and Ad-i satisfy the Formula (7) . If YES, the process goes 
to Step 324 in which the representative value A d +i in the 
period L d+i is selected. If NO, the process goes to Step 
330 in which the end of matching is determined. 
[0064] In Step 324, the representative value A d+ i in the 
quantization period L d+1 at the data-base side is selected 
and, in Step 325, a determination is made as to whether A 6 
and A d+ i satisfy the Formula (7) . if YES, the process goes 
to Step 326 in which the result of matching is outputted. 
If NO, the process goes to Step 331 in which the end of 
matching is determined. 

[0065] In Step 326, the relevant data based on the re- 
sult of matching is derived from the data-base and is out- 
putted. 

[0066] In Step 327 through Step 329, a determination is 
made as to whether the next quantization period L d , L d -i or 
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L<i+i exists or not. If YES, the process goes to Step 316 in 
which the matching operation is continued for the next L^. 
If NO, the process goes to Step 326 in which the matching 
result as to whether the Formula is satisfied is output- 
ted. 

[0067] In Step 330 through Step 331, a determination is 
made as to whether the next representative value A^-i or 
A d +i exists or not. If YES, the process goes to Step 316 in 
which the matching operation is continued for the next Ld- 
If NO, the process goes to Step 326 in which the matching 
result is outputted, 

[0068] As explained above, according to the present 
embodiment, by giving changing width or allowance to the 
quantization width T and the quantization period length L 
as well as the representative values at the respective 
quantization periods, it is possible to perform the search 
for the movie image information even in the case where the 
image size of the input image which is inputted as the 
search process condition is different from the image size 
of the image at the data-base side, or in the case where 
the coding rate which is one factor to determine the 
amount of information is incorrect in the image compres- 
sion technique which is used for reducing the amount of 
information of the image data* Further, here again, as 
explained before, in the matching of the feature value 
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information between the input side and the data-base side, 
all the steps are not necessarily performed* Matching re- 
sult up to the intermediate step may well be used, if nec- 
essary. 

[0069] As explained hereinabove, according to the in- 
vention, because the amount of data used as input is re- 
duced, the problem in that the load on the hardware be- 
comes large is solved, and the shortening of time required 
for the search process is achieved. 

[0070] While the invention has been described in its 
preferred embodiments, it is to be understood that the 
words which have been used are words of description rather 
than limitation and that changes within the purview of the 
appended claims may be made without departing from the 
scope of the invention as defined by the claims. 



